{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!guardrails hub install hub://guardrails/similar_to_document"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Summarize text accurately\n",
    "\n",
    "!!! note\n",
    "    To download this example as a Jupyter notebook, click [here](https://github.com/ShreyaR/guardrails/blob/main/docs/examples/text_summarization_quality.ipynb).\n",
    "\n",
    "In this example, we will use Guardrails in the summarization of a text document. We will check whether the summarized document has a high semantic similarity with the original document.\n",
    "\n",
    "## Objective\n",
    "\n",
    "Summarize a text document and check whether the summarized document has a high semantic similarity with the original document.\n",
    "\n",
    "## Step 0: Setup\n",
    "\n",
    "In order to run this example, you will need to install the `numpy` package. You can do so by running the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy in /home/zayd/workspace/shreya/guardrails/.venv/lib/python3.11/site-packages (1.25.2)\n"
     ]
    }
   ],
   "source": [
    "!pip install numpy"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Create the RAIL Spec\n",
    "\n",
    "Ordinarily, we would create an RAIL spec in a separate file. For the purposes of this example, we will create the spec in this notebook as a string following the RAIL syntax. For more information on RAIL, see the [RAIL documentation](/docs/how_to_guides/rail).  We will also show the same RAIL spec in a code-first format using a Pydantic model.\n",
    "\n",
    "In this RAIL spec, we:\n",
    "\n",
    "1. Create an `output` schema that returns a single key-value pair. The key should be 'summary', and the value should be the summary of the given document."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First let's open our document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('data/article1.txt', 'r') as file:\n",
    "    document = file.read()\n",
    "    file.seek(0)\n",
    "    content = ''.join(line.strip() for line in file.readlines())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we can define our RAIL spec either as a XML string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from string import Template\n",
    "\n",
    "rail_str = Template(\"\"\"\n",
    "<rail version=\"0.1\">\n",
    "\n",
    "<output>\n",
    "    <string\n",
    "        name=\"summary\"\n",
    "        description=\"Summarize the given document faithfully.\"\n",
    "        format=\"similar-to-document: {${document}}, 0.60\"\n",
    "        on-fail-similar-to-document=\"filter\" \n",
    "    />\n",
    "</output>\n",
    "\n",
    "<prompt>\n",
    "Summarize the following document:\n",
    "\n",
    "${document}\n",
    "\n",
    "${gr.complete_json_suffix}\n",
    "</prompt>\n",
    "</rail>\n",
    "\"\"\").safe_substitute(document=document)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or as a Pydantic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "from guardrails.hub import SimilarToDocument\n",
    "\n",
    "prompt = \"\"\"\n",
    "Summarize the following document:\n",
    "\n",
    "${document}\n",
    "\n",
    "${gr.complete_json_suffix}\n",
    "\"\"\"\n",
    "\n",
    "class DocumentSummary(BaseModel):\n",
    "    summary: str = Field(\n",
    "        description=\"Summarize the given document faithfully.\",\n",
    "        validators=[SimilarToDocument(document=f\"'{content}'\", threshold=0.60, on_fail=\"filter\")]\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "!!! note\n",
    "\n",
    "    In order to ensure the summary is similar to the document, we use `similar-to-document` as the validator. This validator embeds the document and the summary and checks whether the cosine similarity between the two embeddings is above a threshold."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Create a `Guard` object with the RAIL Spec\n",
    "\n",
    "We create a `gd.Guard` object that will check, validate and correct the output of the LLM. This object:\n",
    "\n",
    "1. Enforces the quality criteria specified in the RAIL spec.\n",
    "2. Takes corrective action when the quality criteria are not met.\n",
    "3. Compiles the schema and type info from the RAIL spec and adds it to the prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import guardrails as gd\n",
    "\n",
    "from rich import print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From our RAIL string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "guard = gd.Guard.from_rail_string(rail_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or from our Pydantic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "guard = gd.Guard.from_pydantic(output_class=DocumentSummary, prompt=prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see the prompt that will be sent to the LLM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">\n",
       "Summarize the following document:\n",
       "\n",
       "$<span style=\"font-weight: bold\">{</span>document<span style=\"font-weight: bold\">}</span>\n",
       "\n",
       "\n",
       "Given below is XML that describes the information to extract from this document and the tags to extract it into.\n",
       "\n",
       "<span style=\"font-weight: bold\">&lt;</span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff; font-weight: bold\">output</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">    &lt;string </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"summary\"</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #808000; text-decoration-color: #808000\">description</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"Summarize the given document faithfully.\"</span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">&lt;</span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">output</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;</span>\n",
       "\n",
       "\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">ONLY return a valid JSON object </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span><span style=\"color: #000000; text-decoration-color: #000000\">no other text is necessary</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span><span style=\"color: #000000; text-decoration-color: #000000\">, where the key of the field in JSON is the `name` </span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">attribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The JSON</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and </span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">specific types. Be correct and concise. If you are unsure anywhere, enter `null`.</span>\n",
       "\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">Here are examples of simple </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span><span style=\"color: #000000; text-decoration-color: #000000\">XML, JSON</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span><span style=\"color: #000000; text-decoration-color: #000000\"> pairs that show the expected behavior:</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">- `&lt;string </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'foo'</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #808000; text-decoration-color: #808000\">format</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'two-words lower-case'</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;` =&gt; `</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'foo'</span><span style=\"color: #000000; text-decoration-color: #000000\">: </span><span style=\"color: #008000; text-decoration-color: #008000\">'example one'</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">}</span><span style=\"color: #000000; text-decoration-color: #000000\">`</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">- `&lt;list </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'bar'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;&lt;string </span><span style=\"color: #808000; text-decoration-color: #808000\">format</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'upper-case'</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;&lt;</span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">list</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;` =&gt; `</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">\"bar\"</span><span style=\"color: #000000; text-decoration-color: #000000\">: </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">[</span><span style=\"color: #008000; text-decoration-color: #008000\">'STRING ONE'</span><span style=\"color: #000000; text-decoration-color: #000000\">, </span><span style=\"color: #008000; text-decoration-color: #008000\">'STRING TWO'</span><span style=\"color: #000000; text-decoration-color: #000000\">, etc.</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">]}</span><span style=\"color: #000000; text-decoration-color: #000000\">`</span>\n",
       "<span style=\"color: #000000; text-decoration-color: #000000\">- `&lt;object </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'baz'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;&lt;string </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"foo\"</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #808000; text-decoration-color: #808000\">format</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"capitalize two-words\"</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;&lt;integer </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"index\"</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #808000; text-decoration-color: #808000\">format</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">\"1-indexed\"</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span>\n",
       "<span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;&lt;</span><span style=\"color: #800080; text-decoration-color: #800080\">/</span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff\">object</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;` =</span><span style=\"font-weight: bold\">&gt;</span> `<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'baz'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'foo'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Some String'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'index'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span><span style=\"font-weight: bold\">}}</span>`\n",
       "\n",
       "\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\n",
       "Summarize the following document:\n",
       "\n",
       "$\u001b[1m{\u001b[0mdocument\u001b[1m}\u001b[0m\n",
       "\n",
       "\n",
       "Given below is XML that describes the information to extract from this document and the tags to extract it into.\n",
       "\n",
       "\u001b[1m<\u001b[0m\u001b[1;95moutput\u001b[0m\u001b[39m>\u001b[0m\n",
       "\u001b[39m    <string \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"summary\"\u001b[0m\u001b[39m \u001b[0m\u001b[33mdescription\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"Summarize\u001b[0m\u001b[32m the given document faithfully.\"\u001b[0m\u001b[35m/\u001b[0m\u001b[39m>\u001b[0m\n",
       "\u001b[39m<\u001b[0m\u001b[35m/\u001b[0m\u001b[95moutput\u001b[0m\u001b[39m>\u001b[0m\n",
       "\n",
       "\n",
       "\u001b[39mONLY return a valid JSON object \u001b[0m\u001b[1;39m(\u001b[0m\u001b[39mno other text is necessary\u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m, where the key of the field in JSON is the `name` \u001b[0m\n",
       "\u001b[39mattribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The JSON\u001b[0m\n",
       "\u001b[39mMUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and \u001b[0m\n",
       "\u001b[39mspecific types. Be correct and concise. If you are unsure anywhere, enter `null`.\u001b[0m\n",
       "\n",
       "\u001b[39mHere are examples of simple \u001b[0m\u001b[1;39m(\u001b[0m\u001b[39mXML, JSON\u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m pairs that show the expected behavior:\u001b[0m\n",
       "\u001b[39m- `<string \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'foo'\u001b[0m\u001b[39m \u001b[0m\u001b[33mformat\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'two-words lower-case'\u001b[0m\u001b[39m \u001b[0m\u001b[35m/\u001b[0m\u001b[39m>` => `\u001b[0m\u001b[1;39m{\u001b[0m\u001b[32m'foo'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'example one'\u001b[0m\u001b[1;39m}\u001b[0m\u001b[39m`\u001b[0m\n",
       "\u001b[39m- `<list \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'bar'\u001b[0m\u001b[39m><string \u001b[0m\u001b[33mformat\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'upper-case'\u001b[0m\u001b[39m \u001b[0m\u001b[35m/\u001b[0m\u001b[39m><\u001b[0m\u001b[35m/\u001b[0m\u001b[95mlist\u001b[0m\u001b[39m>` => `\u001b[0m\u001b[1;39m{\u001b[0m\u001b[32m\"bar\"\u001b[0m\u001b[39m: \u001b[0m\u001b[1;39m[\u001b[0m\u001b[32m'STRING ONE'\u001b[0m\u001b[39m, \u001b[0m\u001b[32m'STRING TWO'\u001b[0m\u001b[39m, etc.\u001b[0m\u001b[1;39m]\u001b[0m\u001b[1;39m}\u001b[0m\u001b[39m`\u001b[0m\n",
       "\u001b[39m- `<object \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'baz'\u001b[0m\u001b[39m><string \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"foo\"\u001b[0m\u001b[39m \u001b[0m\u001b[33mformat\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"capitalize\u001b[0m\u001b[32m two-words\"\u001b[0m\u001b[39m \u001b[0m\u001b[35m/\u001b[0m\u001b[39m><integer \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"index\"\u001b[0m\u001b[39m \u001b[0m\u001b[33mformat\u001b[0m\u001b[39m=\u001b[0m\u001b[32m\"1\u001b[0m\u001b[32m-indexed\"\u001b[0m\u001b[39m \u001b[0m\n",
       "\u001b[35m/\u001b[0m\u001b[39m><\u001b[0m\u001b[35m/\u001b[0m\u001b[95mobject\u001b[0m\u001b[39m>` =\u001b[0m\u001b[1m>\u001b[0m `\u001b[1m{\u001b[0m\u001b[32m'baz'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'foo'\u001b[0m: \u001b[32m'Some String'\u001b[0m, \u001b[32m'index'\u001b[0m: \u001b[1;36m1\u001b[0m\u001b[1m}\u001b[0m\u001b[1m}\u001b[0m`\n",
       "\n",
       "\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(guard.base_prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, `statement_to_be_translated` is the the statement and will be provided by the user at runtime."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Wrap the LLM API call with `Guard`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's try translating a statement that doesn't have any profanity in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Async event loop found, but guard was invoked synchronously.For validator parallelization, please call `validate_async` instead.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Validated Output: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'summary'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'The US Congress consists of a Senate and House of Representatives, with the House of</span>\n",
       "<span style=\"color: #008000; text-decoration-color: #008000\">Representatives being chosen every two years by the people of the several states. Representatives must be at least </span>\n",
       "<span style=\"color: #008000; text-decoration-color: #008000\">25 years old and have been a citizen of the US for seven years. Representation and taxes are apportioned among the </span>\n",
       "<span style=\"color: #008000; text-decoration-color: #008000\">states according to their population, and the number of representatives cannot exceed one for every 30,000 people. </span>\n",
       "<span style=\"color: #008000; text-decoration-color: #008000\">Vacancies are filled by the executive authority of the state. The House of Representatives chooses its speaker and </span>\n",
       "<span style=\"color: #008000; text-decoration-color: #008000\">other officers, and has the sole power of impeachment.'</span><span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "Validated Output: \u001b[1m{\u001b[0m\u001b[32m'summary'\u001b[0m: \u001b[32m'The US Congress consists of a Senate and House of Representatives, with the House of\u001b[0m\n",
       "\u001b[32mRepresentatives being chosen every two years by the people of the several states. Representatives must be at least \u001b[0m\n",
       "\u001b[32m25 years old and have been a citizen of the US for seven years. Representation and taxes are apportioned among the \u001b[0m\n",
       "\u001b[32mstates according to their population, and the number of representatives cannot exceed one for every 30,000 people. \u001b[0m\n",
       "\u001b[32mVacancies are filled by the executive authority of the state. The House of Representatives chooses its speaker and \u001b[0m\n",
       "\u001b[32mother officers, and has the sole power of impeachment.'\u001b[0m\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import openai\n",
    "\n",
    "raw_llm_response, validated_response, *rest = guard(\n",
    "    openai.completions.create,\n",
    "    prompt_params={'document': document},\n",
    "    model='text-davinci-003',\n",
    "    max_tokens=2048,\n",
    "    temperature=0\n",
    ")\n",
    "\n",
    "print(f\"Validated Output: {validated_response}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to see a detailed look into the logs of the `Guard` object, we can print the `Guard` state history:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Logs\n",
       "└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮\n",
       "    │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Summarize the following document:                                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Section. 1.                                                                                             │ │\n",
       "    │ │ All legislative Powers herein granted shall be vested in a Congress of the United States, which shall   │ │\n",
       "    │ │ consist of a Senate and House of Representatives.                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Section. 2.                                                                                             │ │\n",
       "    │ │ The House of Representatives shall be composed of Members chosen every second Year by the People of the │ │\n",
       "    │ │ several States, and the Electors in each State shall have the Qualifications requisite for Electors of  │ │\n",
       "    │ │ the most numerous Branch of the State Legislature.                                                      │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ No Person shall be a Representative who shall not have attained to the Age of twenty five Years, and    │ │\n",
       "    │ │ been seven Years a Citizen of the United States, and who shall not, when elected, be an Inhabitant of   │ │\n",
       "    │ │ that State in which he shall be chosen.                                                                 │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Representatives and direct Taxes shall be apportioned among the several States which may be included    │ │\n",
       "    │ │ within this Union, according to their respective Numbers, which shall be determined by adding to the    │ │\n",
       "    │ │ whole Number of free Persons, including those bound to Service for a Term of Years, and excluding       │ │\n",
       "    │ │ Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three │ │\n",
       "    │ │ Years after the first Meeting of the Congress of the United States, and within every subsequent Term of │ │\n",
       "    │ │ ten Years, in such Manner as they shall by Law direct. The Number of Representatives shall not exceed   │ │\n",
       "    │ │ one for every thirty Thousand, but each State shall have at Least one Representative; and until such    │ │\n",
       "    │ │ enumeration shall be made, the State of New Hampshire shall be entitled to chuse three, Massachusetts   │ │\n",
       "    │ │ eight, Rhode-Island and Providence Plantations one, Connecticut five, New-York six, New Jersey four,    │ │\n",
       "    │ │ Pennsylvania eight, Delaware one, Maryland six, Virginia ten, North Carolina five, South Carolina five, │ │\n",
       "    │ │ and Georgia three.                                                                                      │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ When vacancies happen in the Representation from any State, the Executive Authority thereof shall issue │ │\n",
       "    │ │ Writs of Election to fill such Vacancies.                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "    │ │ Power of Impeachment.                                                                                   │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "    │ │ it into.                                                                                                │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ &lt;output&gt;                                                                                                │ │\n",
       "    │ │     &lt;string name=\"summary\" description=\"Summarize the given document faithfully.\"/&gt;                     │ │\n",
       "    │ │ &lt;/output&gt;                                                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "    │ │ - `&lt;string name='foo' format='two-words lower-case' /&gt;` =&gt; `{'foo': 'example one'}`                     │ │\n",
       "    │ │ - `&lt;list name='bar'&gt;&lt;string format='upper-case' /&gt;&lt;/list&gt;` =&gt; `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "    │ │ etc.]}`                                                                                                 │ │\n",
       "    │ │ - `&lt;object name='baz'&gt;&lt;string name=\"foo\" format=\"capitalize two-words\" /&gt;&lt;integer name=\"index\"          │ │\n",
       "    │ │ format=\"1-indexed\" /&gt;&lt;/object&gt;` =&gt; `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Json Output:                                                                                            │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "    │ │ ┏━━━━━━┳━━━━━━━━━┓                                                                                      │ │\n",
       "    │ │ ┃<span style=\"font-weight: bold\"> Role </span>┃<span style=\"font-weight: bold\"> Content </span>┃                                                                                      │ │\n",
       "    │ │ ┡━━━━━━╇━━━━━━━━━┩                                                                                      │ │\n",
       "    │ │ └──────┴─────────┘                                                                                      │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "    │ │ {\"summary\": \"The US Congress consists of a Senate and House of Representatives, with the House of       │ │\n",
       "    │ │ Representatives being chosen every two years by the people of the several states. Representatives must  │ │\n",
       "    │ │ be at least 25 years old and have been a citizen of the US for seven years. Representation and taxes    │ │\n",
       "    │ │ are apportioned among the states according to their population, and the number of representatives       │ │\n",
       "    │ │ cannot exceed one for every 30,000 people. Vacancies are filled by the executive authority of the       │ │\n",
       "    │ │ state. The House of Representatives chooses its speaker and other officers, and has the sole power of   │ │\n",
       "    │ │ impeachment.\"}                                                                                          │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "    │ │ {                                                                                                       │ │\n",
       "    │ │     'summary': 'The US Congress consists of a Senate and House of Representatives, with the House of    │ │\n",
       "    │ │ Representatives being chosen every two years by the people of the several states. Representatives must  │ │\n",
       "    │ │ be at least 25 years old and have been a citizen of the US for seven years. Representation and taxes    │ │\n",
       "    │ │ are apportioned among the states according to their population, and the number of representatives       │ │\n",
       "    │ │ cannot exceed one for every 30,000 people. Vacancies are filled by the executive authority of the       │ │\n",
       "    │ │ state. The House of Representatives chooses its speaker and other officers, and has the sole power of   │ │\n",
       "    │ │ impeachment.'                                                                                           │ │\n",
       "    │ │ }                                                                                                       │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "Logs\n",
       "└── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮\n",
       "    │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Summarize the following document:                                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Section. 1.                                                                                             │ │\n",
       "    │ │ All legislative Powers herein granted shall be vested in a Congress of the United States, which shall   │ │\n",
       "    │ │ consist of a Senate and House of Representatives.                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Section. 2.                                                                                             │ │\n",
       "    │ │ The House of Representatives shall be composed of Members chosen every second Year by the People of the │ │\n",
       "    │ │ several States, and the Electors in each State shall have the Qualifications requisite for Electors of  │ │\n",
       "    │ │ the most numerous Branch of the State Legislature.                                                      │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ No Person shall be a Representative who shall not have attained to the Age of twenty five Years, and    │ │\n",
       "    │ │ been seven Years a Citizen of the United States, and who shall not, when elected, be an Inhabitant of   │ │\n",
       "    │ │ that State in which he shall be chosen.                                                                 │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Representatives and direct Taxes shall be apportioned among the several States which may be included    │ │\n",
       "    │ │ within this Union, according to their respective Numbers, which shall be determined by adding to the    │ │\n",
       "    │ │ whole Number of free Persons, including those bound to Service for a Term of Years, and excluding       │ │\n",
       "    │ │ Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three │ │\n",
       "    │ │ Years after the first Meeting of the Congress of the United States, and within every subsequent Term of │ │\n",
       "    │ │ ten Years, in such Manner as they shall by Law direct. The Number of Representatives shall not exceed   │ │\n",
       "    │ │ one for every thirty Thousand, but each State shall have at Least one Representative; and until such    │ │\n",
       "    │ │ enumeration shall be made, the State of New Hampshire shall be entitled to chuse three, Massachusetts   │ │\n",
       "    │ │ eight, Rhode-Island and Providence Plantations one, Connecticut five, New-York six, New Jersey four,    │ │\n",
       "    │ │ Pennsylvania eight, Delaware one, Maryland six, Virginia ten, North Carolina five, South Carolina five, │ │\n",
       "    │ │ and Georgia three.                                                                                      │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ When vacancies happen in the Representation from any State, the Executive Authority thereof shall issue │ │\n",
       "    │ │ Writs of Election to fill such Vacancies.                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "    │ │ Power of Impeachment.                                                                                   │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "    │ │ it into.                                                                                                │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ <output>                                                                                                │ │\n",
       "    │ │     <string name=\"summary\" description=\"Summarize the given document faithfully.\"/>                     │ │\n",
       "    │ │ </output>                                                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "    │ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`                     │ │\n",
       "    │ │ - `<list name='bar'><string format='upper-case' /></list>` => `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "    │ │ etc.]}`                                                                                                 │ │\n",
       "    │ │ - `<object name='baz'><string name=\"foo\" format=\"capitalize two-words\" /><integer name=\"index\"          │ │\n",
       "    │ │ format=\"1-indexed\" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Json Output:                                                                                            │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "    │ │ ┏━━━━━━┳━━━━━━━━━┓                                                                                      │ │\n",
       "    │ │ ┃\u001b[1m \u001b[0m\u001b[1mRole\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mContent\u001b[0m\u001b[1m \u001b[0m┃                                                                                      │ │\n",
       "    │ │ ┡━━━━━━╇━━━━━━━━━┩                                                                                      │ │\n",
       "    │ │ └──────┴─────────┘                                                                                      │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "    │ │ {\"summary\": \"The US Congress consists of a Senate and House of Representatives, with the House of       │ │\n",
       "    │ │ Representatives being chosen every two years by the people of the several states. Representatives must  │ │\n",
       "    │ │ be at least 25 years old and have been a citizen of the US for seven years. Representation and taxes    │ │\n",
       "    │ │ are apportioned among the states according to their population, and the number of representatives       │ │\n",
       "    │ │ cannot exceed one for every 30,000 people. Vacancies are filled by the executive authority of the       │ │\n",
       "    │ │ state. The House of Representatives chooses its speaker and other officers, and has the sole power of   │ │\n",
       "    │ │ impeachment.\"}                                                                                          │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "    │ │ {                                                                                                       │ │\n",
       "    │ │     'summary': 'The US Congress consists of a Senate and House of Representatives, with the House of    │ │\n",
       "    │ │ Representatives being chosen every two years by the people of the several states. Representatives must  │ │\n",
       "    │ │ be at least 25 years old and have been a citizen of the US for seven years. Representation and taxes    │ │\n",
       "    │ │ are apportioned among the states according to their population, and the number of representatives       │ │\n",
       "    │ │ cannot exceed one for every 30,000 people. Vacancies are filled by the executive authority of the       │ │\n",
       "    │ │ state. The House of Representatives chooses its speaker and other officers, and has the sole power of   │ │\n",
       "    │ │ impeachment.'                                                                                           │ │\n",
       "    │ │ }                                                                                                       │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(guard.history.last.tree)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `guard` wrapper returns the raw_llm_respose (which is a simple string), and the validated and corrected output (which is a dictionary). We can see that the output is a dictionary with the correct schema and types.\n",
    "\n",
    "Next, let's try using a smaller model, which is not going to be good at summarization. We can see that the output is filtered out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Validated Output: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "Validated Output: \u001b[3;35mNone\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "raw_llm_response, validated_response, *rest = guard(\n",
    "    openai.completions.create,\n",
    "    prompt_params={'document': open(\"data/article1.txt\", \"r\").read()},\n",
    "    model='text-ada-001',\n",
    "    max_tokens=512,\n",
    "    temperature=0\n",
    ")\n",
    "\n",
    "print(f\"Validated Output: {validated_response}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the step-wise history of the `Guard` object below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Logs\n",
       "├── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮\n",
       "│   │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Summarize the following document:                                                                       │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Section. 1.                                                                                             │ │\n",
       "│   │ │ All legislative Powers herein granted shall be vested in a Congress of the United States, which shall   │ │\n",
       "│   │ │ consist of a Senate and House of Representatives.                                                       │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Section. 2.                                                                                             │ │\n",
       "│   │ │ The House of Representatives shall be composed of Members chosen every second Year by the People of the │ │\n",
       "│   │ │ several States, and the Electors in each State shall have the Qualifications requisite for Electors of  │ │\n",
       "│   │ │ the most numerous Branch of the State Legislature.                                                      │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ No Person shall be a Representative who shall not have attained to the Age of twenty five Years, and    │ │\n",
       "│   │ │ been seven Years a Citizen of the United States, and who shall not, when elected, be an Inhabitant of   │ │\n",
       "│   │ │ that State in which he shall be chosen.                                                                 │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Representatives and direct Taxes shall be apportioned among the several States which may be included    │ │\n",
       "│   │ │ within this Union, according to their respective Numbers, which shall be determined by adding to the    │ │\n",
       "│   │ │ whole Number of free Persons, including those bound to Service for a Term of Years, and excluding       │ │\n",
       "│   │ │ Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three │ │\n",
       "│   │ │ Years after the first Meeting of the Congress of the United States, and within every subsequent Term of │ │\n",
       "│   │ │ ten Years, in such Manner as they shall by Law direct. The Number of Representatives shall not exceed   │ │\n",
       "│   │ │ one for every thirty Thousand, but each State shall have at Least one Representative; and until such    │ │\n",
       "│   │ │ enumeration shall be made, the State of New Hampshire shall be entitled to chuse three, Massachusetts   │ │\n",
       "│   │ │ eight, Rhode-Island and Providence Plantations one, Connecticut five, New-York six, New Jersey four,    │ │\n",
       "│   │ │ Pennsylvania eight, Delaware one, Maryland six, Virginia ten, North Carolina five, South Carolina five, │ │\n",
       "│   │ │ and Georgia three.                                                                                      │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ When vacancies happen in the Representation from any State, the Executive Authority thereof shall issue │ │\n",
       "│   │ │ Writs of Election to fill such Vacancies.                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "│   │ │ Power of Impeachment.                                                                                   │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "│   │ │ it into.                                                                                                │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ &lt;output&gt;                                                                                                │ │\n",
       "│   │ │     &lt;string name=\"summary\" description=\"Summarize the given document faithfully.\"/&gt;                     │ │\n",
       "│   │ │ &lt;/output&gt;                                                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "│   │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "│   │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "│   │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "│   │ │ enter `null`.                                                                                           │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "│   │ │ - `&lt;string name='foo' format='two-words lower-case' /&gt;` =&gt; `{'foo': 'example one'}`                     │ │\n",
       "│   │ │ - `&lt;list name='bar'&gt;&lt;string format='upper-case' /&gt;&lt;/list&gt;` =&gt; `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "│   │ │ etc.]}`                                                                                                 │ │\n",
       "│   │ │ - `&lt;object name='baz'&gt;&lt;string name=\"foo\" format=\"capitalize two-words\" /&gt;&lt;integer name=\"index\"          │ │\n",
       "│   │ │ format=\"1-indexed\" /&gt;&lt;/object&gt;` =&gt; `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Json Output:                                                                                            │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "│   │ │ ┏━━━━━━┳━━━━━━━━━┓                                                                                      │ │\n",
       "│   │ │ ┃<span style=\"font-weight: bold\"> Role </span>┃<span style=\"font-weight: bold\"> Content </span>┃                                                                                      │ │\n",
       "│   │ │ ┡━━━━━━╇━━━━━━━━━┩                                                                                      │ │\n",
       "│   │ │ └──────┴─────────┘                                                                                      │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "│   │ │     &lt;string name=\"summary\" description=\"Summarize the given document faithfully.\"/&gt;                     │ │\n",
       "│   │ │ &lt;/string&gt;                                                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "│   │ │ Power of Impeachment.                                                                                   │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "│   │ │ None                                                                                                    │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "└── ╭────────────────────────────────────────────────── Step 1 ───────────────────────────────────────────────────╮\n",
       "    │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ I was given the following response, which was not parseable as JSON.                                    │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ {                                                                                                       │ │\n",
       "    │ │   \"incorrect_value\": \"    &lt;string name=\\\"summary\\\" description=\\\"Summarize the given document           │ │\n",
       "    │ │ faithfully.\\\"/&gt;\\n&lt;/string&gt;\\n\\nThe House of Representatives shall chuse their Speaker and other          │ │\n",
       "    │ │ Officers; and shall have the sole Power of Impeachment.\",                                               │ │\n",
       "    │ │   \"error_messages\": [                                                                                   │ │\n",
       "    │ │     \"Output is not parseable as JSON\"                                                                   │ │\n",
       "    │ │   ]                                                                                                     │ │\n",
       "    │ │ }                                                                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Help me correct this by making it valid JSON.                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "    │ │ it into.                                                                                                │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ &lt;output&gt;                                                                                                │ │\n",
       "    │ │     &lt;string name=\"summary\" description=\"Summarize the given document faithfully.\"/&gt;                     │ │\n",
       "    │ │ &lt;/output&gt;                                                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Json Output:                                                                                            │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ You are a helpful assistant only capable of communicating with valid JSON, and no other text.           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "    │ │ - `&lt;string name='foo' format='two-words lower-case' /&gt;` =&gt; `{'foo': 'example one'}`                     │ │\n",
       "    │ │ - `&lt;list name='bar'&gt;&lt;string format='upper-case' /&gt;&lt;/list&gt;` =&gt; `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "    │ │ etc.]}`                                                                                                 │ │\n",
       "    │ │ - `&lt;object name='baz'&gt;&lt;string name=\"foo\" format=\"capitalize two-words\" /&gt;&lt;integer name=\"index\"          │ │\n",
       "    │ │ format=\"1-indexed\" /&gt;&lt;/object&gt;` =&gt; `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "    │ │ No message history.                                                                                     │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "    │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "    │ │ Power of Impeachment.                                                                                   │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "    │ │ None                                                                                                    │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "Logs\n",
       "├── ╭────────────────────────────────────────────────── Step 0 ───────────────────────────────────────────────────╮\n",
       "│   │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Summarize the following document:                                                                       │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Section. 1.                                                                                             │ │\n",
       "│   │ │ All legislative Powers herein granted shall be vested in a Congress of the United States, which shall   │ │\n",
       "│   │ │ consist of a Senate and House of Representatives.                                                       │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Section. 2.                                                                                             │ │\n",
       "│   │ │ The House of Representatives shall be composed of Members chosen every second Year by the People of the │ │\n",
       "│   │ │ several States, and the Electors in each State shall have the Qualifications requisite for Electors of  │ │\n",
       "│   │ │ the most numerous Branch of the State Legislature.                                                      │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ No Person shall be a Representative who shall not have attained to the Age of twenty five Years, and    │ │\n",
       "│   │ │ been seven Years a Citizen of the United States, and who shall not, when elected, be an Inhabitant of   │ │\n",
       "│   │ │ that State in which he shall be chosen.                                                                 │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Representatives and direct Taxes shall be apportioned among the several States which may be included    │ │\n",
       "│   │ │ within this Union, according to their respective Numbers, which shall be determined by adding to the    │ │\n",
       "│   │ │ whole Number of free Persons, including those bound to Service for a Term of Years, and excluding       │ │\n",
       "│   │ │ Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three │ │\n",
       "│   │ │ Years after the first Meeting of the Congress of the United States, and within every subsequent Term of │ │\n",
       "│   │ │ ten Years, in such Manner as they shall by Law direct. The Number of Representatives shall not exceed   │ │\n",
       "│   │ │ one for every thirty Thousand, but each State shall have at Least one Representative; and until such    │ │\n",
       "│   │ │ enumeration shall be made, the State of New Hampshire shall be entitled to chuse three, Massachusetts   │ │\n",
       "│   │ │ eight, Rhode-Island and Providence Plantations one, Connecticut five, New-York six, New Jersey four,    │ │\n",
       "│   │ │ Pennsylvania eight, Delaware one, Maryland six, Virginia ten, North Carolina five, South Carolina five, │ │\n",
       "│   │ │ and Georgia three.                                                                                      │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ When vacancies happen in the Representation from any State, the Executive Authority thereof shall issue │ │\n",
       "│   │ │ Writs of Election to fill such Vacancies.                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "│   │ │ Power of Impeachment.                                                                                   │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "│   │ │ it into.                                                                                                │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ <output>                                                                                                │ │\n",
       "│   │ │     <string name=\"summary\" description=\"Summarize the given document faithfully.\"/>                     │ │\n",
       "│   │ │ </output>                                                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "│   │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "│   │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "│   │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "│   │ │ enter `null`.                                                                                           │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "│   │ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`                     │ │\n",
       "│   │ │ - `<list name='bar'><string format='upper-case' /></list>` => `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "│   │ │ etc.]}`                                                                                                 │ │\n",
       "│   │ │ - `<object name='baz'><string name=\"foo\" format=\"capitalize two-words\" /><integer name=\"index\"          │ │\n",
       "│   │ │ format=\"1-indexed\" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ Json Output:                                                                                            │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "│   │ │ ┏━━━━━━┳━━━━━━━━━┓                                                                                      │ │\n",
       "│   │ │ ┃\u001b[1m \u001b[0m\u001b[1mRole\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mContent\u001b[0m\u001b[1m \u001b[0m┃                                                                                      │ │\n",
       "│   │ │ ┡━━━━━━╇━━━━━━━━━┩                                                                                      │ │\n",
       "│   │ │ └──────┴─────────┘                                                                                      │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "│   │ │     <string name=\"summary\" description=\"Summarize the given document faithfully.\"/>                     │ │\n",
       "│   │ │ </string>                                                                                               │ │\n",
       "│   │ │                                                                                                         │ │\n",
       "│   │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "│   │ │ Power of Impeachment.                                                                                   │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "│   │ │ None                                                                                                    │ │\n",
       "│   │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "│   ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "└── ╭────────────────────────────────────────────────── Step 1 ───────────────────────────────────────────────────╮\n",
       "    │ ╭──────────────────────────────────────────────── Prompt ─────────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ I was given the following response, which was not parseable as JSON.                                    │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ {                                                                                                       │ │\n",
       "    │ │   \"incorrect_value\": \"    <string name=\\\"summary\\\" description=\\\"Summarize the given document           │ │\n",
       "    │ │ faithfully.\\\"/>\\n</string>\\n\\nThe House of Representatives shall chuse their Speaker and other          │ │\n",
       "    │ │ Officers; and shall have the sole Power of Impeachment.\",                                               │ │\n",
       "    │ │   \"error_messages\": [                                                                                   │ │\n",
       "    │ │     \"Output is not parseable as JSON\"                                                                   │ │\n",
       "    │ │   ]                                                                                                     │ │\n",
       "    │ │ }                                                                                                       │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Help me correct this by making it valid JSON.                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Given below is XML that describes the information to extract from this document and the tags to extract │ │\n",
       "    │ │ it into.                                                                                                │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ <output>                                                                                                │ │\n",
       "    │ │     <string name=\"summary\" description=\"Summarize the given document faithfully.\"/>                     │ │\n",
       "    │ │ </output>                                                                                               │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Json Output:                                                                                            │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭───────────────────────────────────────────── Instructions ──────────────────────────────────────────────╮ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ You are a helpful assistant only capable of communicating with valid JSON, and no other text.           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the │ │\n",
       "    │ │ `name` attribute of the corresponding XML, and the value is of the type specified by the corresponding  │ │\n",
       "    │ │ XML's tag. The JSON MUST conform to the XML format, including any types and format requests e.g.        │ │\n",
       "    │ │ requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere,     │ │\n",
       "    │ │ enter `null`.                                                                                           │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ │ Here are examples of simple (XML, JSON) pairs that show the expected behavior:                          │ │\n",
       "    │ │ - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`                     │ │\n",
       "    │ │ - `<list name='bar'><string format='upper-case' /></list>` => `{\"bar\": ['STRING ONE', 'STRING TWO',     │ │\n",
       "    │ │ etc.]}`                                                                                                 │ │\n",
       "    │ │ - `<object name='baz'><string name=\"foo\" format=\"capitalize two-words\" /><integer name=\"index\"          │ │\n",
       "    │ │ format=\"1-indexed\" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`                        │ │\n",
       "    │ │                                                                                                         │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Message History ────────────────────────────────────────────╮ │\n",
       "    │ │ No message history.                                                                                     │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭──────────────────────────────────────────── Raw LLM Output ─────────────────────────────────────────────╮ │\n",
       "    │ │ The House of Representatives shall chuse their Speaker and other Officers; and shall have the sole      │ │\n",
       "    │ │ Power of Impeachment.                                                                                   │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    │ ╭─────────────────────────────────────────── Validated Output ────────────────────────────────────────────╮ │\n",
       "    │ │ None                                                                                                    │ │\n",
       "    │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │\n",
       "    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(guard.history.last.tree)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "guardrails",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
